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ABSTRACT 



Several groups have examined gender- related issues affecting 
initial entry into training in the Department of Defense. The studies that 
have gotten the most attention are the Army's Senior Review Panel's report on 
sexual harassment, the report of the Federal Advisory Commission on 
Gender- Integrated Training and Related Issues, and the report of the Defense 
Advisory Committee on Women in the Services. Because the groups produced 
somewhat different results, this study was conducted to: (1) describe how 
they conducted their studies; (2) determine how well the work supported 
making conclusions and recommendations; (3) assess the availability of 
documentation supporting the report; and (4) determine the extent to which 
the final report described the study methodology and disclosed limitations. 
Each of the studies used focus groups as the primary data gathering method or 
in conjunction with other data collection means. For this reason, the 
principles of focus group methodology set forth in social sciences literature 
were considered. The Army's Senior Review Panel's study most closely followed 
the methodology for conducting focus groups recommended in the literature. In 
addition, the use of multiple methods of data gathering, including site 
visits, interviews, and surveys of 22,952 individuals and the publication of 
this data provided ample support for making conclusions and recommendations. 
The Federal Advisory Committee on Gender- Integrated Training used focus 
groups, but did not systematically collect the same information from all 
groups, document the information generated, or explain how what was heard in 
focus groups and interviews led to their conclusions. The extent to which the 
Committee's work supports its conclusions cannot be determined. The Defense 
Advisory Committee on Women in the Services used focus groups that were 
sometimes too large, or too brief for meaningful participation, and they did 
not document the results on a group -by-group basis. This Committee summarized 
the opinion and perception data, but did not make recommendations or draw 
conclusions. One appendix discusses focus group methodology, and the other 
gives a summary of each of the three reports. (SLD) 
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Over the past year, several groups have examined gender-related issues 
affecting initial entry training in the Department of Defense (dod). The 
studies that have gotten the most attention are the Army’s Senior Review 
Panel’s report on sexual harassment, the report of the Federal Advisory 
Committee on Gender-Integrated Training and Related Issues, and the 
report of the Defense Advisory Committee on Women in the Services. 
Because the groups produced somewhat different results, our objectives 
were to (1) describe how the groups conducted their work, (2) determine 
how well the work supported making conclusions and recommendations, 

(3) assess the availability of documentation supporting the report, and 

(4) determine the extent to which the final report described the study 
methodology and disclosed limitations. 

Since each of the three studies used focus groups as either their primary 
data gathering method or in concert with other means of collecting 
information, we used social science literature on focus groups to assess 
their use of that methodology. Focus groups use a small group discussion 
format, guided by a moderator, to gather information on a given topic. 
Focus groups are useful in providing an understanding of the depth and 
the variety of opinions, feelings, or beliefs, but are not useful for 
quantifying the extent to which a particular view or opinion is held. For 
these reasons, focus groups are a qualitative technique best used when the 
aim is to explore an issue or, at the other end of the research process, to 
assess reaction to specific proposals. As with any methodology, its value is 
dependent upon the extent to which it is appropriate to the objectives of 
the inquiry, how well the methodology is conducted, and the way in which 
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the results are used. More information on focus group methodology is 
found in appendix I. 



Background 



In November 1996, the Secretary of the Army established the Senior 
Review Panel on Sexual Harassment. The panel’s mission was to make 
recommendations to improve the human relations environment in which 
soldiers live and work, with the specific goal of eradicating sexual 
harassment in the Army. The panel consisted of seven members, including 
two retired general officers recalled to active duty, two active duty general 
officers, a senior noncommissioned officer, and two dod civilians. The 
Senior Review Panel forwarded its report 1 and recommendations to the 
Secretary of the Army in July 1997. It included 40 recommendations, of 
which 14 dealt with training and related issues. 

In June 1997, the Secretary of Defense established the Federal Advisory 
Committee on Gender-Integrated Training and Related Issues. The 
Secretary of Defense established the Committee as a result of the sexual 
misconduct incidents at Aberdeen Proving Grounds. Former Senator 
Nancy Kassebaum Baker chaired a panel of 11 that included civilians, 
retired officers, and a retired senior noncommissioned officer. The 
Secretary directed the Committee to assess the training programs and 
policies of the Army, Navy, Air Force, and Marine Corps and make 
recommendations to improve initial entry training. The Committee issued 
its report 2 to the Secretary of Defense on December 16, 1997. It made 30 
recommendations covering the full cycle from recruitment through basic 
and advanced training. 

The Defense Advisory Committee on Women in the Services (dacowits) 
has been advising secretaries of Defense since George Marshall 
established the Committee in 1951. dacowits, which consists of 30 to 40 
civilians, makes recommendations to the Secretary on the roles of women 
in the Armed Forces and on quality of life issues affecting readiness. As 



‘An Army summary of the findings of this report appears in appendix II. 

-A DOD summary of the report’s key recommendations and findings appears in appendix II. 



The reports of sexual misconduct at Aberdeen Proving Grounds led the 
Secretary of Defense to establish the Federal Advisory Committee on 
Gender-Integrated Training and Related Issues and to ask the Defense 
Advisory Committee on Women in the Services to meet with trainees and 
trainers. These incidents also prompted the Secretary of the Army to 
establish the Senior Review Panel on Sexual Harassment. 




Page 2 



GAO/NSIAD-98-125 Gender Issues 



B-279460 



part of its mission, dacowits members conduct annual visits to selected 
Army, Air Force, Navy, Marine Corps, and Coast Guard installations, both 
here and overseas. These visits serve two purposes: (1) to provide the 
Secretary of Defense with insight into the thoughts and perceptions of 
servicemembers in the fleet and the field and (2) to determine what issues 
dacowits will concentrate on in the future. In November 1996, the former 
Secretary of Defense requested that dacowits visit training installations to 
meet with trainees and trainers in the training environment. In 
February 1997, the current Secretary of Defense endorsed the request. 
dacowits provided a report 3 to the Secretary of Defense summarizing these 
visits. In its report, dacowits recommended continued visits to training 
installations, but made no recommendations on military training. 



Results in Brief 



The three studies have different objectives and were conducted somewhat 
differently. We compared the methodologies of the three studies with the 
principles of focus group methodologies set forth in social science 
literature. We found a wide variance among the three studies. The Army’s 
Senior Review Panel’s study most closely followed the methodology for 
conducting focus groups recommended by the literature. Specifically, we 
found: 

The Army’s Senior Review Panel on Sexual Harassment used four methods 
to collect data: individual interviews, focus groups, surveys, and 
observations. During its 8 months of work, the panel visited 59 
installations worldwide, conducted interviews with 808 military and 
civilian Army personnel, ran focus groups with over 8,000 soldiers and 
civilians, and surveyed 22,952 individuals. The use of multiple methods of 
data gathering, the rigor with which the various methods were conducted, 
and the publication of the data in the report provides ample support for 
making conclusions and recommendations. 

The Federal Advisory Committee on Gender-Integrated Training and 
Related Issues used focus groups as its primary method of data gathering. 
Although the FAC conducted over 300 focus groups and individual 
interviews, their value for making conclusions and recommendations is 
limited because the Committee did not (1) systematically collect the same 
information from all groups, (2) document the information generated in 
each of the interviews and focus groups, or (3) explain how what was 
heard in the interviews and focus groups led to their conclusions and 
recommendations. In addition, the length of the focus group sessions, the 



3 A summary of the findings excerpted from the report’s executive summary can be found in 
appendix II. 
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number of participants, and the number of questions addressed may not 
have provided adequate time for full participation of the respondents on 
all issues. Given these limitations, the extent to which the Committee’s 
work supports its conclusions and recommendations cannot be 
determined. 

• The Defense Advisory Committee on Women in the Services also used 
focus groups of trainees, trainers, and supervisors in the Army, Air Force, 
Navy, Marine Corps, and Coast Guard to identify what issues concerned 
women and men at training installations. Members of dacowits held focus 
group discussions at 12 schools at 9 installations in the United States and 
prepared a summary report of the results at each installation. The 
dacowits Chair used these to prepare a report to the Secretary of Defense 
that accurately reflected the opinions and perceptions cited in the 
individual installation reports. We noted that the dacowits focus groups 
were (1) larger than recommended in the literature, (2) were sometimes 
not long enough to allow meaningful participation, and (3) were not 
recorded or documented on a group-by-group basis. The dacowits report 
summarized the opinion and perception data obtained from focus groups. 
It made no conclusions or recommendations on military training based on 
that information. 



The Army’s Senior 
Review Panel on 
Sexual Harassment 



The Army s Senior Review Panel on Sexual Harassment formed four 
teams, one to review Army policies and three for data collection. Each 
field team consisted, on the average, of six military personnel and one 
civilian. The Chair, the Vice-Chair, or the Deputy Assistant Secretary of the 
Army (a member of the panel) accompanied each field team during their 
visits. Other panel members traveled with the teams as often as possible. 

Visits lasted 1 to 4 days depending on the numbers of participants in the 
various activities. Before the visits, the participants for the individual 
interviews, focus groups, and survey were selected and scheduled. 
Generally, the visit started with a briefing to present the purpose of the 
activity and a description of the team’s data collection efforts. Next, the 
team divided into smaller groups to conduct individual interviews, 
conduct focus groups, or administer surveys. These activities ran 
concurrently and team members rotated to different activities at different 
times. Visits ended with a briefing providing commanders the opportunity 
to begin corrective actions. 

Panel members and the working group collaborated in writing the panel’s 
report. Once a near final draft was generated, the panel members met for a 
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final review and agreement on the content. The panel on Sexual 
Harassment issued its report to the Secretary of the Army in July 1997. 



Methodology 



The Army’s Senior Review Panel on Sexual Harassment used four methods 
to collect data: individual interviews, focus groups, surveys, and 
observations. According to the senior social scientist detailed to the panel, 
the field teams conducted interviews and focus groups using carefully 
developed protocols to obtain human relations environment information. 
Members of field teams conducted individual interviews with 808 military 
and civilian Army leaders and personnel in Army support groups. Focus 
groups consisted of randomly selected, single-gender groups of 8 to 12 
people organized by ranks or categories. 4 Participants totaled 7,401 
soldiers and 1,007 civilians. Facilitators and note takers of the same 
gender as the groups conducted the sessions. All data obtained through 
these two activities were entered into a computer for analysis. The 
working group, which consisted of more than 40 military and civilian 
personnel, developed main themes or categories and placed the 
perceptions data under the categories. Data were then analyzed by rank, 
by gender, and by question. 

The written surveys addressed leadership, cohesion, and sexual 
harassment. Field teams administered the surveys to 22,952 
servicemembers. Surveys were developed for trainees, trainers, and the 
general Army population. The working group analyzed survey data using a 
standard statistical analysis software package. Statistically projectable 
results appear in the report by question and, in some cases, by gender. 
Observations were made during visits to barracks and other facilities and 
to engage in informal conversations with military and civilian personnel, 
family members, and others. 

The seven panel members, supported by the working group, collected data 
at 59 Army installations worldwide selected using a stratified random 
sampling design. Stratification was based on the type and location of the 
installation. The study took 8 months to complete and obtained 
information from over 32,000 Army personnel. 



4 Rank or category included, junior enlisted soldiers, junior noncommissioned officers, senior 
noncommissioned officers, company grade officers, field grade officers, civilian employees, drill 
sergeants, instructors, and trainees. 
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Use of Multiple 
Methodologies and 
Extensive Documentation 
Allowed Making 
Conclusions and 
Recommendations 



Lack of Survey Pretest Results 
in Controversy 



ERIC 



The panel’s methodology supported making conclusions and 
recommendations. Focus groups were used in cor\junction with surveys to 
not only confirm the survey data but also provide texture and perspective 
to the data. The focus groups were of an appropriate size and were all 
asked the same questions, in the same order by trained moderators. 
However, the number of questions asked of many of the focus groups was 
significantly greater than the five or six questions recommended by focus 
group literature. For example, the set of questions for trainee focus groups 
consisted of 15 questions and the set for trainers consisted of 13 questions. 
Focus group discussions were not tape recorded because it was feared 
that this would inhibit the participants, but notes were taken by a note 
taker and were content-analyzed. The notes from each focus group session 
were destroyed, after the responses were entered in the database and 
verified for accuracy, to assure that participant confidentiality was 
maintained. Destroying the original documentation to assure 
confidentiality is considered an appropriate measure by social scientists. 

In addition, the completed survey forms were also destroyed to assure 
participant confidentiality. 

In volume two, the panel provides an extensive discussion of its 
methodology. Volume two provides details on how participants were 
selected, copies of the focus group questions, the surveys, and the 
interview questions. Results of the surveys were included in the report as 
well as the most frequently heard responses in the focus groups. 
Furthermore, a sufficient amount of data are presented in volume one of 
the report, which outlines the panel’s conclusions and recommendations, 
to allow the reader to evaluate them. 

An area of controversy arose because the survey developers did not 
pretest the survey questions. We were told by the senior social scientist 
attached to the panel, that tight time frames precluded the panel from 
carrying out a pretest of the survey form. Normally, a pretest is performed 
to identify problem questions, problems with language interpretations, 
unclear instructions, or to determine if there are some questions that 
respondents will refuse to answer. In this instance, the survey form 
contained six questions that some respondents in early administrations 
found inflammatory, offensive, and an invasion of privacy and resulted in 
some refusals to complete the survey. Subsequently, those questions were 
eliminated and a revised form was used. The data on the six questions 
were not included in the database, which resulted in an accusation that the 
panel had eliminated important data from its analysis. The report 
disclosed the problem and its resolution in the methodology section. We 
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believe that the panel acted responsibly in eliminating the offending 
questions to avoid having a negative effect on the survey return rate. The 
controversy, however, demonstrates the importance of pretesting survey 
forms before conducting a survey. 



The Federal Advisory 
Committee on 
Gender-Integrated 
Training and Related 
Issues 



The Federal Advisory Committee on Gender-Integrated Training and 
Related Issues saw its role as listening to the views of trainees, trainers, 
supervisors, and service officials and providing the Secretary of Defense 
with its best judgment about what should be done to improve training. 
Small teams of Committee members visited 17 training installations and 
operational units to gather opinions. Most Committee members visited 
installations from two services. While the Committee Chairman visited 
installations for all four services, no Committee member or Committee 
staff member visited all of the installations . 



Once at an installation, the Committee members followed the same 
general schedule: reveille, breakfast with new servicemembers, meetings 
with command officials, and interviews and focus groups before lunch. 
After lunch with support personnel, the Committee members conducted 
additional focus groups and interviews. At the end of each visit, they met 
with command officials to discuss their findings. Installation visits 
generally lasted 1 day, although visits to basic training sites were 2-day 
trips. The visits to the training installations and operational units occurred 
in September and October 1997. 

The Committee had two public meetings, the first in July 1997 and the 
second in October 1997. At the July meeting, service representatives 
provided information on the services’ recruiting and training programs. At 
the October 1997 meeting, Committee members discussed their 
observations and agreed to a partial list of recommendations for the 
report. The Committee’s staff drafted the report based on the discussions 
they heard during their installation visits and the public meeting, and 
memorandums submitted by some of the Committee members in 
preparation for the October meeting. Committee members received the 
draft report in early December and revisions were made based on their 
comments. The Committee Chair discussed the report with Committee 
members in a series of one-on-one telephone calls to arrive at the final 
recommendations. 
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Methodology 



Problems With the 
Methodology Limit the 
Usefulness of the Report 



The Committee s primary means of collecting information involved focus 
group discussions. The Committee held 199 focus groups, soliciting 
opinions from more than 1,000 trainees, 500 trainers, 300 first-term 
servicemembers, and 275 supervisors at U.S. training installations and 
operational units. Participants were randomly selected under the 
supervision of the installations’ inspectors general. Trainees who 
participated in the focus groups were within 2 weeks of completing their 
training. Participants in the trainer focus groups were trainers for at least 1 
year. First term participants were in their initial assignment and had been 
on the job between 6 and 18 months. Generally, the Committee met with 
equal numbers of females and males, although, because of the limited 
number of female trainers and supervisors, this was not always possible. 

Focus groups included about 10 to 15 people each and were 
gender-segregated. All of the focus groups were moderated by Committee 
members, and generally two Committee members or a Committee member 
and a Committee staff member attended each session. The Committee 
members worked from a set of questions tailored for each service and 
each type of focus group. Although the number of questions varied by type 
of focus group, the set of questions for all basic training focus groups 
consisted of 20 questions, some of which had multiple parts. While some 
focus groups were scheduled to last only 30 to 45 minutes, most focus 
group sessions lasted nearly an hour. 

Committee members also conducted over 100 interviews with service 
officials, including commanding officers, inspectors general, company or 
squadron commanders, and senior noncommissioned officers. They also 
met with representatives of support groups such as chaplains, equal 
opportunity officers, medical officers, and legal officers. 



The value of the information included in the Committee’s report for 
making conclusions and recommendations is limited because the 
Committee did not follow recommended focus group methodology. The 
Committee believed that a more flexible approach to the discussions 
would enhance the quality of the exchange between the participants and 
the Committee members. However, the fact that the same questions were 
not asked of each similar focus group, along with the number of questions, 
size of the groups, and length of the sessions may have combined to limit 
full discussion. In addition, the focus groups’ discussions were not 
systematically recorded. As a result, the extent to which the 
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recommendations are supported by the Committee’s work cannot be 
assessed. 

The Committee staff provided the Committee members with questions for 
the focus groups. However, according to the staff director, the Committee 
members were told that the questions were guidelines and that they did 
not have to be asked as written. Because the Committee members had the 
flexibility to ask any question they desired, the responses should not be 
compared with each other. Also, the number of questions provided to the 
panel members were far more than the five to six focus group literature 
recommends. For example, the staff provided 20 questions, some of which 
had several parts, for the Committee to ask Army trainees at Fort Leonard 
Wood, Missouri. Fifteen trainees participated in the 1-hour focus groups at 
this installation. If the entire hour was spent on the questions, there would 
have only been 3 minutes spent on each question and only 15 seconds for 
each participant to respond. We do not believe that would have been 
enough time for a meaningful discussion of a question. Finally, even if all 
the questions were asked as they were written, they were not always 
asked in the same order each time. Social science literature suggests that 
the same questions asked in a different sequence may result in different 
responses. 

The absence of documentation of the comments made in the individual 
focus groups was the most serious methodological shortcoming. While the 
Committee members took notes during each focus group, these notes 
were not made part of the Committee’s records nor were they summarized 
and included in the report. Without documentation, it is impossible to 
determine if the Committee’s work supports its recommendations: Also, 
the lack of documentation prevented the Committee from analyzing the 
data to know what comments they heard or how often similar comments 
were made. Knowing how often a particular kind of comment was made 
and the subgroup of the person who made it are ways of putting the 
comments in perspective and filtering biases. 

The report of the Federal Advisory Committee on Gender-Integrated 
Training and Related Issues does not include a sufficient discussion of the 
Committee’s methodology and work process. For example, the report 
states that the Committee conducted discussion groups with randomly 
selected servicemembers, but it does not explain the random selection 
process. In addition, the use of terminology such as “randomly selected” 
implies a level of scientific rigor that was not achieved in this study. The 
report does not identify the make-up of the discussion groups, discuss 
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what type of data analysis was done or not done, or mention any 
limitations with the data. Limitations that we believe should have been 
mentioned are that the report was based on opinions and the results 
cannot be generalized to the entire military training population. 

Also, the report often presents opinions in a manner that they can be 
misinterpreted as facts based on empirical data. For example, the report 
says that the Committee members observed that integrated housing is 
contributing to a higher rate of disciplinary problems, but, according to the 
Chairman, the Committee did not obtain any data to support this 
statement. In addition, the report contains many statements that include 
words like “most”, “many”, and “majority”. These words lead a reader to 
believe that the Committee counted responses to particular questions or 
polled the focus group participants. The Chairman said that the Committee 
does not have quantitative data. 

The report also does not explain the process the Committee used to 
formulate its recommendations. Although the Committee held a public 
meeting in October 1997 after its installation visits had been completed, 
the recommendations on separate barracks for male and female recruits 
and on the organization of gender-segregated platoons, divisions, and 
flights were not made until after that meeting. Furthermore, those 
recommendations were not discussed by the Committee as a whole, but 
rather in a series of calls to individual Committee members. 



The Defense Advisory 
Committee on Women 
in the Services 



The mission of the dacowits effort was to provide the Secretary of Defense 
with an overview of broad issues raised by trainees and trainers of both 
genders throughout initial entry training. A secondary purpose was to help 
determine what issues dacowits would concentrate on in the future. The 
Chair and the Executive Director of dacowits selected seven members (all 
were women) to visit training installations. Members were selected based 
on their dacowits experience and the quantity and quality of their previous 
installation reports. Typical visits were conducted by one dacowits 
member and lasted 2 days. Visits began with a briefing by the commanding 
officer about the school and its mission, followed by trainee, trainer, and 
supervisor focus groups. At the end of a visit, the dacowits member met 
with command officials to share the results of the focus groups. Reports, 
summarizing the most frequently heard comments from the various focus 
groups, were written at the Conclusion of each visit. In addition, the seven 
members met at dacowits’ 1997 fall conference to discuss the results of 
their visits. Using the reports and the conference discussion, the 1997 
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dacowits Chair wrote the report. The report was released by the Secretary 
of Defense in January 1998. 


Methodology 


dacowits used focus groups as its primary means of data gathering. 

Overall, they solicited the opinions of over 1,200 trainees, trainers, and 
supervisors in the Army, the Navy, the Marines, the Air Force, and the 
Coast Guard in focus group discussions at 12 gender-integrated training 
schools at 9 installations. The schools included enlisted basic, 
intermediate, and advance training, and officer advanced training. Most 
focus groups were gender-segregated and trainees, trainers, and 
supervisors were in separate focus groups as well, dacowits requested 
trainees with at least 40 percent of training completed. Many trainees had 
completed their training and were awaiting graduation. 

The groups averaged 20 participants and sessions lasted about 60 minutes, 
although some were shorter. Before meeting with the Committee 
members, focus group participants viewed an 18-minute video that 
explained the mission of dacowits and highlighted some of the gender 
equality issues that dacowits had worked on in the past such as sexual 
harassment, discrimination, child care, and the combat exclusion policy. 
The video set the stage for the two open-ended questions that all the 
participants were asked: (1) “How is it going?” and (2) “If you had five 
minutes to speak with the Secretary of Defense, what would you tell him?” 
According to the former Chair, dacowits uses these questions during all 
installation visits. Training installation visits took place between July and 
November 1997. 

At the conclusion of each visit, a dacowits member completed a 
standardized installation visit report summarizing the most frequently 
heard comments from the focus groups. The comments included in these 
reports were entered into a computer and sorted by frequency across the 
services as well as by individual service. Issues were included in the report 
to the Secretary based on frequency. The individual installation visit 
reports support the opinions and perceptions that appear in the report to 
the Secretary of Defense. 


Problems With the 
Methodology Limit the 
Usefulness of the Report 


Some focus groups may have been too large or may not have had enough 
time to allow ample participation by most of the participants. The 
literature suggests that focus groups should be no larger than 
12 participants. During the dacowits visits to the training schools, some 
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groups were as large as 20 participants. Groups larger than 12 usually do 
not allow sufficient opportunity to actively participate in the discussion 
and are more difficult to manage. Also, the majority of the sessions were 
about an hour long and some ran for only 45 minutes, about half the time 
recommended by focus group literature. 



dacowits used two questions to generate discussion. However, time may 
still have been a problem, since the questions were very open-ended and 
could be taken in virtually any direction by a participant. This would likely 
have the effect of increasing the amount of time needed as each 
participant not only answered the discussion questions, but also reacted 
and responded to the issues raised by others. 

dacowits did not document the individual focus groups as recommended 
by focus group literature. Instead, dacowits members prepared installation 
visit reports which summarized the opinions they heard most frequently. 
While the installation reports document the work performed and the 
issues surfaced during the training installation visits, they do not capture 
enough information about the discussions in each focus group to be really 
useful. For example, they do not provide enough information on the rank 
or gender of the groups that raised the issue which would help put the 
comments into perspective. 

As we stated earlier, all of the dacowits members making installation 
training installation visits were women. Some focus group literature 
suggests that the gender of the moderator and the gender of the focus 
group should be the same, particularly when the issues being discussed 
are sensitive or have a direct bearing on the opposite sex. Also, some 
focus group literature suggests that men are more likely to tell a woman 
moderator what they think will impress or please her rather than what 
they think. The use of female moderators for male focus groups, in 
conjunction with the women’s advocacy impression that the video is likely 
to have conveyed, may have made some males hesitant to raise issues or 
perceptions that might be construed as anti-female. Because dacowits did 
not document each of its focus groups it is impossible to determine if the 
use of women moderators with all-male focus groups had an effect on the 
responses of the male participants. 

The dacowits report provides some methodological information for the 
reader, but does not provide some key information. First, the report does 
not provide any details on how the Committee members documented the 
focus groups. Second, the report does not clearly explain the process used 
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by dacowits to determine what issues would be included in the report. 
Third, while the report provides some detail about the make-up of the 
focus groups it does not describe how the focus group participants were 
selected. It should be noted however, as recommended by focus group 
literature, the report clearly states its two major limitations: (1) the 
opinion and perception information included in the report has not been 
independently validated or confirmed and ( 2) the Committee did not visit 
any gender-segregated training facilities. Also, in accordance with the 
limitations of the methodology, the dacowits report made no conclusions 
or recommendations on military training. 



Agency Comments 



We provided a draft of this report to dod, the Chairman of the Federal 
Advisory Committee on Gender-Integrated Training and Related Issues, 
and the former Chair and Military Director of dacowits for comment. 

We discussed our report with Department of the Army officials, who 
concurred with our observations on the Army’s Senior Review Panel on 
Sexual Harassment. We also discussed the draft report with the Executive 
Director of the Federal Advisory Committee on Gender-Integrated 
Training and Related Issues who suggested some clarifications to the 
report, which we considered and made as appropriate. In addition, we 
discussed the draft with the military director of dacowits, who stated that 
dacowits does not aim to meet the standards of academic research but 
instead uses focus groups to collect opinions and identify issues for 
further study. Finally, we discussed the draft with the former Chair of 
dacowits who suggested some technical corrections which we made as 
appropriate. 



Scope and 
Methodology 



We reviewed the reports from the Army’s Senior Review Panel on Sexual 
Harassment, the Federal Advisory Committee on Gender-Integrated 
Training and Related Issues, and dacowits. We reviewed literature on the 
conduct and use of focus groups, since that was a common methodology 
across the three studies. We focused on the methodological information 
provided in the reports, including any limitations on the use of the 
information. We reviewed supporting documents to determine if the 
evidence collected supports making conclusions and recommendations. 
We did not evaluate the validity of specific conclusions and 
recommendations made by any of the studies. 
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We met with the Chairman and Executive Director of the Federal Advisory 
Committee on Gender-Integrated Training and Related Issues, the former 
Chair and Military Director of dacowits, and with the senior social scientist 
of the Army’s Senior Review Panel on Sexual Harassment to thoroughly 
explore the approach and methodology used in these efforts. 

Our review was requested by the former Ranking Minority Member of the 
House National Security Committee and Mr. Meehan. We are addressing 
the report to the current Ranking Minority Member of the House National 
Security Committee, Mr. Skelton, as a courtesy. We are addressing this 
letter to Senator Robb because it is related to other work on gender issues 
in the military that we have undertaken at his request. 

We conducted our review in February and March 1998 in accordance with 
generally accepted government auditing standards. 



We are sending copies of this report to the Secretaries of Defense, the 
Army, the Air Force, and the Navy; the Chairman of the Joint Chiefs of 
Staff; and the Commandant of the Marine Corps. We will make copies 
available to any other interested parties. 

The major contributors to this report were Carol R. Schuster, William E. 
Beusse, Carole F. Coffey, George M. Delgado, and Kathleen M. Joyce. If 
you or your staff have any questions concerning this report, please call me 
on (202) 512-5140. 




Mark E. Gebicke 
Director, Military Operations 
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Focus Group Methodology 



Focus groups 1 are carefully planned small group discussions involving 
people with similar characteristics who are knowledgeable about an issue 
but do not know each other well. The views expressed in focus groups are 
not necessarily representative of a population and statistical estimates 
cannot be derived from the results. Furthermore, focus groups cannot be 
used to determine the extent of a problem. Focus groups produce 
qualitative data that provide insights into attitudes, perceptions, and 
opinions of the participants. They are most often used before, during, or 
after quantitative research procedures such as surveys. For example, focus 
groups can be used before a survey is undertaken to help a research team 
learn about the target audience or determine the appropriateness of the 
questionnaire. Focus groups are often used with surveys to confirm 
findings and to obtain greater breadth and depth of information. Finally, 
focus groups are often used as a follow-up to surveys to help interpret 
responses. On occasion, focus groups jure used alone when opinions and 
perceptions are more important than how many people hold such views. 

The size of the focus group is an important, but often overlooked, element 
of a successful group discussion. The literature on focus groups suggests 
that an appropriate size for a focus group is 6 to 12 people. A focus group 
with fewer than six participants sometimes has problems with 
productivity because the group has fewer experiences to share. Also, small 
groups can be more easily affected by people who know each other, by 
uncooperative participants, or. by participants who view themselves as 
experts on the topic. Groups that have more than 12 people usually do not 
allow people sufficient opportunity to actively participate in the 
discussion, making the groups difficult to manage. 

The composition of the focus group is also important. Participants should 
share some similar characteristics but be diverse enough to allow for 
differences of opinions. The topic of discussion and the information to be 
obtained dictate the types of characteristics shared. However, generally 
participants should be similar in age, occupation, education, and social 
class. Focus groups with distinct differences among participants such as 
trainees and trainers or junior and senior enlisted personnel do not work 
well because of limited understanding of other lifestyles and situations. 
Furthermore, some participants may be inhibited and defer to those they 
believe to be better educated or more experienced or of a higher social 



'This section drew from the following: Richard A. Krueger, Focus Groups: A Practical Guide for 
Applied Research (Newbury Park, California: Sage Publications, 1988); David L. Morgan, Focus Groups 
As Qualitative Research (Newbury Park, California: Sage Publications, 1988); and David W. Stewart 
and Prem N. Shamdasani, Focus Groups: Theory and Practice (Newbury Park, California: Sage 
Publications, 1990) 
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class. Sometimes, the gender of participants can affect the outcome of a 
focus group and some social scientists recommend against mixing genders 
because men and women tend to perform for each other. When the 
opinions of disparate groups are needed, focus group literature 
recommends holding separate groups for each distinct group. 

Focus group discussions are conducted informally and guided by trained 
moderators who encourage participants to share their thoughts and 
experiences. Trained, experienced moderators are critical to the success 
of a focus group. An unqualified moderator can easily undermine the 
reliability and validity of focus group findings. Successful moderators are 
good listeners, who can make people feel relaxed and anxious to talk. 
Moderators must control a group without being obvious and be aware of 
time. Since literature suggests that focus groups should be scheduled for 
90 minutes and run no more than 120 minutes, moderators need to be able 
to keep the discussion on track and move the participants from one topic 
to the next. Moderators should be aware of the influence that they have on 
the type and amount of data obtained. Moderators must be aware of their 
own biases that might affect the validity of the data and take care not to 
provide cues to participants about desirable responses and answers. If 
dealing with sensitive subjects where views could vary according to 
factors such as gender or race, it is recommended that the moderator be 
similar in gender or race to the participants. Finally, moderators must have 
sufficient knowledge of the topic to put comments in perspective and 
followup on critical areas of concern. 

Questions are the heart of the focus group discussion. The literature on 
focus groups suggests five or six questions for a discussion group. The 
questions need to be carefully thought-out and phrased to result in the 
maximum amount of information in the limited time available. Questions 
should not suggest potential answers and yes or no questions should be 
avoided. Questions should be asked in the same order in every focus group 
and questions should be sequenced from most important to least 
important to ensure that the most necessary information is obtained from 
the participants if time runs out. The sequence is important because the 
questions may interact with one another to form the stimulus that 
generates the responses. If the questions are asked in a different order at 
each focus group, the stimulus is changed and the response will be 
different. 

The results of the focus groups’ discussions should be documented on a 
session by session basis. Focus group literature agrees that the best way to 
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do that is by tape recording supplemented with written notes. However, if 
a tape recording is not feasible or inhibiting to the participants, note taking 
can be sufficient provided they are complete enough to be analyzed. 

A systematic analysis of focus group data is also important. The analysis 
can be either qualitative or quantitative, but it must be systematic and 
verifiable. It must be systematic in that it follows a documented 
step-by-step process and verifiable to permit others to arrive at similar 
conclusions using available documents and the raw results. Social 
scientists have noted that there is a tendency for novice researchers to see 
selectively only those parts of the discussion that confirms their particular 
point of view. Often, a researcher will go into the discussion with certain 
hunches of how participants might feel. As a result, the researcher tends to 
look for evidence to support these hunches and overlook data that present 
different points of view. A systematic and verifiable process helps 
researchers in filtering out bias and assuring that they present the data as 
objectively as possible. 

Once data are collected and analyzed, the data should be reported and, if 
appropriate, conclusions and recommendations made. A report should 
clearly state what the purpose of the study was, what its scope was, how 
the data were collected and analyzed, and what, if any, significant 
limitations exist on the data or the use of the data. For example, studies 
that used focus groups as the primary method of data collection should 
clearly state that the data being reported is opinion or perception. If the 
opinions have been substantiated by other types of data, this should be 
clearly stated in the report. The report should also include the results of 
the focus groups, and the results should be clearly stated so that a reader 
can come to the same conclusions as the report writers. 
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The Secretary of the 
Army’s Senior Review 
Panel on Sexual 
Harassment 



The following is excerpted from Army press reports that accompanied the 
report of the Senior Review Panel’s report on Sexual Harassment in the 
Army as well as the executive summary of the report: 

The panel found: 

• Sexual harassment exists throughout the Army, crossing gender, rank, and 
racial lines; gender discrimination is more common than sexual 
harassment. 

• Army leaders are the critical factor in creating, maintaining and enforcing 
an environment of respect and dignity in the Army; too many leaders have 
failed to gain the trust of their soldiers. 

• The Army lacks institutional commitment to the Equal Opportunity 
program and soldiers distrust the equal opportunity program. 

• Trainees believe the overwhelming majority of drill sergeants and 
instructors perform competently and well, but “respect” as an Army core 
value is not well institutionalized in the Initial Entry training process. 

Recommendations of the panel were broad-based and covered a wide 
variety of Army processes including: leader development, equal 
opportunity policy and procedures, initial entry training soldierization, 
unit and institutional training, command climate, and oversight. 



Federal Advisory 
Committee on 
Gender-Integrated Training 
and Related Issues 



The following is an excerpt from the press release that accompanied the 
report of the Federal Advisory Committee on Gender-Integrated Training 
and Related Issues which delivered its report to the Secretary of Defense 
in December 1997: 

“The panel studied the full training cycle including recruiting, basic training, and advanced 
skills training. Its recommendations covered the training cadre, housing of recruits, fitness 
programs and follow-on advance training. Among the several recommendations made for 
recruiting, the panel proposed better preparing recruits mentally and physical for basic 
training. It also recommended ways to improve the training cadre. It recommended that 
physical training requirements be toughened and made more uniform throughout the 
services. The panel also suggested that emphasis on discipline be carried over from basic 
to advance training. The panel recommended that value training be incorporated into all 
initial entry training programs and that training get more resources. 



“During visits to training installations, the panel concluded that men and women should be 
housed in separate barracks and train separately at the operational unit level — the Army 
platoon, the Navy division and the Air Force flight. In the Marine Corps men and women 
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live, eat, and train separately. The panel recommended that gender-integrated training 
continue for field training, technical training and classroom work.” 



Defense Advisory The following is an extract from the executive summary of the report to 

Committee on Women in the Secretary of Defense from the Defense Advisory Committee on Women 
the Services in the Services: 



The scope of dacowits training installation visits included all elements of initial entry 
training, including basic training, advanced individual training, and officer advanced 
training. The majority of issues raised by trainees, trainers, and supervisors of trainers 
were similar across all of the Armed Forces. 



The most frequently raised issues by women and men and trainees and trainers alike were 
artificial gender relationships imposed at training installations, the persistence of gender 
discriminatory behaviors at many locations; the relationship between trainer attitudes and 
gender climates; the under valuation of trainers, especially women trainers; the need for 
greater gender integration to train field and fleet ready service members, the need to 
increase physical training opportunities and standards; the need to improve screening of 
new recruits and to harmonize recruiting quality and practices; the under resourcing of 
training schools and the need to improve support services for women trainees.” 
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